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A method allowing to distinguish interacting from non- 
interacting systems based on available time series is proposed 
and investigated. Some facts concerning generalized Renyi 
dimensions that form the basis of our method are proved. 
We show that one can find the dimension of the part of the 
attractor of the system connected with interaction between 
its parts. We use our method to distinguish interacting from 
non-interacting systems on the examples of logistic and Henon 
maps. A classification of all possible interaction schemes is 
given. 
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I. INTRODUCTION 

Given two time series can one tell if they originated 
from interacting or non-interacting systems? We show 
that with the help of embedding methods Q, Takens 
theorem ®^ and some facts concerning Renyi dimen- 
sions which we prove, one can succeed in case of chaotic 
systems. Moreover, one can quantify the common part 
of the dynamics, which we call dynamics of interaction. 

It happens sometimes, especially in simple systems 
like electronic circuits or coupled mechanical oscillators, 
that one knows whether the systems under investigation 
are coupled or not, what is the direction and sometimes 
strength of the coupling. However, there are many com- 
plex phenomena in nature where one is unable to verify 
directly the existence of coupling between parts of the 
system in which the phenomenon takes place. 

Especially in complex spatiotemporal systems, like 
fluid systems, brain, neuronal tissue, social systems etc. 
one often faces the problem of characterization of inter- 
dependence of parts of the system of interest and quan- 
tifying the strength of interactions between the parts. 

Recent research in neurology, for example, has shown 
that temporal coordination between different, often dis- 
tant neural assemblies plays a critical role in the neuro- 
physiological underpinnings of such cognitive phenomena 
as the integration of features in object representation (cf. 
\i\ for a review) and the conscious experience of stimuli 
[5|. The critical empirical question, therefore, is which 
of the neural assemblies synchronize their activity. Since 
coordination may take many forms, including complex 
non-linear relations, simple correlational methods may 



not be sufficient to detect it. The detection of nonlin- 
ear forms of coordination is also critically important for 
issues in cognitive science pj, developmental psychology 
, and social psychology g|| . 

The method traditionally used for this purpose is cor- 
relation analysis. Given two time series one studies their 
autocorrelation functions and cross-correlations. Large 
cross-correlations are usually attributed to large inter- 
dependence between the parts. Small cross-correlations 
are considered as the signature of independence of the 
variables. 

Unfortunately, the linear time series analysis gives 
meaningful results only in case of linear systems or sto- 
chastic time series. It is well-known that spectral analysis 
alone cannot discriminate between low-dimensional non- 
linear deterministic systems and stochastic systems [ [l0| ], 
even though the properties of the two kinds of systems 
are different. 

Methods based on entropy measures represent one vi- 
able approach for detecting nonlinear relations between 
the activity of different neural assemblies ||. 

Recently another approach based on nonlinear mutual 
prediction has been proposed and used in an experiment. 
Pecora, Caroll and Heagy [jllj developed a statistics to 
study the topological nature of functional relationship 
between coupled systems. Schiff et al. |l2j used it as 
a basis of their method. The idea is as follows: if there 
exists a functional relationship between two systems, it 
is possible to predict state of one system from the known 
states of the other. This happens if the coupling between 
two systems is strong enough so that generalized syn- 
chronization occurs |0,|l^,0]. The average normalized 
mutual prediction error is used to quantify the strength 
and directionality of the coupling |T3] . 

The method we introduce in the present paper does not 
assume generalized synchrony. We introduce the notion 
of the dimension of interaction, which measures the size 
of the dynamics responsible for the coupling between the 
two systems. More precisely, it is the dimension of the 
part of the attractor of the whole system, which is acted 
on by the dynamics of both subsystems. We also show 
how to obtain information concerning the strength and 
directionality of the coupling. 

The idea is, in fact, very simple. Given two time se- 
ries from subsystems of interest we construct another one 
which probes the whole system, for instance adding the 
two series. If the subsystems do not interact, dimension 
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of the whole system is the sum of the dimensions of the 
two subsystems, all of which can be estimated from data. 
On the other hand, if the subsystems have some common 
degrees of freedom, dimension of the whole system will 
be smaller than the sum of the dimensions of the two 
subsystems. 

Our method can also be used to find out if two re- 
sponse systems have a common driver. We discuss this 



application in Section III. 

The structure of the paper is as follows. In Section |l| 
we recall the definition of the Renyi dimensions and 
formulate three theorems which form the basis of our 
method. The, rather straightforward, proofs have been 
relegated to Appendix A, since they are not crucial for 
understanding the method itself and can be omitted by 
readers whose main interest is in applications. We for- 
mulate our method in section III. Classification of all 



the possible interaction schemes is given in Section IV 



A simple way of verifying the kind and direction of the 
coupling is provided. Results from the simulations of cou- 
pled logistic and Henon maps are collected in Section [v|. 
Final comments and outlook are given in the last section. 



II. THEORETICAL CONSIDERATIONS 
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and the sum is taken over all cells with pi ^ 0. 

Of particular importance are Do — the box-counting 
dimension, usually equal to the Hausdorff dimension 
the information dimension or the di- 



fl5|,£4|j25|, D 1 

mension of the measure [[19 26 - 2{| , which describes how 
the entropy — J2iPi^°SPi increases with the change of 
the scale, and D 2 — the correlation dimension f2^ , |23| , |30f , 
which can be most easily extracted from data, usually 
treated as a lower estimate of D\ since D qi < D q2 for 

qi > ?2- 

Generalized dimensions are defined for all real q, how- 
ever in proofs we shall restrict our attention to the case 
q > 1. We are particularly interested in q = 1 and q = 2. 



Our method presented in Section III is based on three 



theorems relating dimensions of subsystems to the di- 
mension of the whole system. The first one states the 
intuitively obvious fact that the dimension of a system 
consisting of two non-interacting parts is the sum of the 
dimensions of the subsystems. A less trivial Theorems 2 
and 3 establish interdependencies among the dimensions 
of the system and its interacting parts. Before we state 
our theorems we shall recall the definition of the Renyi 
dimensions. 



B. Non-interacting systems 

Consider two non-interacting dynamical systems 
(Ui,tpi,fjLi), (U 2 ,(p2,fJ>2), where U l C W H is the phase 
space, tfi is a flow or a map on Ui, and [ii is an ergodic 
(pi-invariant natural measure on Ui. 

Below we shall concentrate on the case of continuous 
systems. Changes needed for the discrete time case are 
mostly notational. 

By natural measure [i we mean 



A. Renyi dimensions 

It is at present generally accepted that a lot of ob- 
jects, both in the real physical space and in the phase 
space, are multifractals [[l5|-^8[. This means they can be 
described by (statistically) self-similar probability mea- 
sures. This usually implies that they can be decomposed 
into a (infinite) number of objects of different Haussdorff 
dimensions, or, equivalently, they have non-trivial multi- 
fractal spectra of dimensions. 

The Renyi dimensions Jl9| have drawn attention of 
physicists and mathematicians after publication of the 
papers by Grassberger, Hentschel and Procaccia [53 p3[ . 
For a probability measure /1 on a c£-dimensional space U 
one takes a partition of U into small cells of equal linear 
size e (equal volume e d ). One defines the Renyi dimen- 
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5(x - ip t (x ))dt; 



(2) 



in the weak sense for /z-almost every xq (one typically 
thinks of some physical measure, like Sinai-Ruelle-Bowen 
measure [Q); Pi(Ui) = 1. The limit in the weak sense 
means that if we integrate h(xq) with a continuous func- 
tion / on U the limit (||) exists and is ^(-almost every- 
where independent of xq, or in other words, the average of 
/ along a typical trajectory is independent of the trajec- 
tory, thus time averages are equal to ensemble averages. 



An equivalent description of multifractal measures is f a 
spectrum |^,^J3^]. A thorough discussion of the proper- 
ties of D q and f a spectra falls beyond the scope of this pa- 
per. Some good reviews of these with the discussion of the 
abundant literature on multifractals can be found e.g. in 
[p^|- |l^^ -|39|] . Mathematically precise definitions of multi- 
fractal spectra can be found in [ILTjJlq] . 
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We are interested in such measures that the set of Xq for 
which fi(xo) — fi has a non-zero Lebesgue measure. 

The composite non-interacting system has a product 
structure (U\ x U 2 ,fx x (p 2 ,fi>i x Ma)- Its dynamics can 
be written as 

f ui(i) = ^i(ui(0),i), 
\ u 2 (f) = V2 (ua(0),i). 

Theorem 1 Suppose D q (/ii), D q (fii), D q (ni x /i 2 ) ex- 
ist. Tften 

D q {p.\ x p 2 ) - ArOi) + D q (fj, 2 ). (3) 

This means, as should be intuitively obvious, the di- 
mensions of non-interacting subsystems add up to the 
dimension of the whole system. The proof is given in Ap- 
pendix A. It follows from extensivity of Renyi entropies. 

This is in fact one of the long-standing problems in 
the dimension theory, namely finding the conditions un- 
der which the equality holds for various dimensions for 
arbitrary measures. Some results for Olsen's version of 
multifractal formalism with a discussion of previous re- 
sults can be found in jilj . 

C. Interacting systems 

Take two interacting subsystems U\ and {/ 2 of system 
U. It may happen that all the variables in U\ couple 
with all those in t/ 2 but this is not necessary. For many- 
dimensional systems the structure of the equations of dy- 
namics can be very complicated. 

Consider the following decomposition of variables of 
Ui. Let yi be the largest set of variables in U\ satisfying 
the condition that if you change their state whatsoever, 
it will not influence the future evolution of U?. Similarly 
define y 2 . Put all the other variables of U\ , C/ 2 in vector x. 
They form a dynamical system V — the part of the whole 
system which is responsible for the interaction. Then the 
dynamics of the whole system U can be written as 

x = /(x), 

yi = 3i(x,yi), (4) 
Y2 = g 2 (x,y 2 ). 

Thus the dynamics of the interacting systems U\ and U2 
is formally equivalent to dynamics of three systems: X 
(interaction part) driving Y\ and Y 2 . We pursue this anal- 
ogy deeper in the next section. An example when such 
decomposition arises naturally is given in Appendix B. 

Let jUa, Mi> M2) Mv be natural measures of dynamical 
systems, respectively, U, Ui,U 2 , V. 

Theorem 2 Suppose D\{ni), Di(p, 2 ), Di(nv), 
Di(nu) exist. Then 

Di{nv) < d int := DM + Dxifia) - Di(jtu). (5) 



(We shall call d mt dimension of interaction ). The equality 
holds when yi and y 2 are asymptotically independent. 

Asymptotical independence means essentially lack of 
generalized synchronization between the ys and their 
common driver x. We relegate further discussion to Ap- 
pendix A, where we make this condition precise and show 
where it is needed^]. 

If we think of dimensions as estimates on the number 
of degrees of freedom, the Theorem 2 means intuitively 
that if the system can be considered as composed of inter- 
acting parts, some of the degrees of freedom — perhaps 
even all — are common for both of the parts. Therefore, 
dimension of the whole system is equal to the sum of the 
number of the common degrees of freedom, those degrees 
of freedom which belong to U\ and do not belong to /7 2 , 
and the other way round. Thus if we add the dimensions 
of the subsystems U\ and J7 2 , we count the common de- 
grees of freedom twice. We must therefore subtract them 
if we want to get dimension of the whole system U . 

In the above theorem we show that this intuition can be 
made precise only in case of the information dimension 
Di and with an additional assumption. The notion of 
the dimension of interaction we define in equation (^) is 
crucial for our method. 

In the special case, when one (or both) of Ui — V, (all 
the variables of Ui couple with some of the variables of 
the other subsystem), say C/ 2 = V, we may establish 

Theorem 3 Suppose D q (/j,i), D q (fi 2 ), D q (/j,v) exist 
and k 2 = n 2 . Then 

D q (fi v ) = d' q nt := D q ( Pl ) + D q { m ) - D q (nu). (6) 

The proof is obvious, for in this case C/ 2 = V and 
Ui = U. This also means that the above intuitions in 
this case are precise for arbitrary generalized dimensions 
and no further assumptions are needed. 

The generalized dimensions of interaction d' q l are es- 
timates on the number of effective degrees of freedom 
responsible for the interaction between the parts of the 
system under study. Of most interest are d'"* = d lnt , 
which has the best analytical properties, and d' 2 nt , which 
can be most reliably estimated from data. 

Note that 

max{^ nt ,L> g (^ y )} < min{_D g (^i),L> g (^ 2 )} 

<max{D,(^i),I> 9 (/X2)} (7) 
< D q ( Pu ). 



2 One would like to establish a similar inequality in case of 
other Renyi dimensions, however, in general 

D q {ii v ) + Dq(fiu) - D q (fj,i) - D q (ii 2 ) 

can have arbitrary sign (cf. Appendix A). Nevertheless, we 
expect this difference for typical physical systems to be small 
in comparison with the dimensions involved. 
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Furthermore, for q — 1 one can show 
< DiO*v) < d^. 
We conjecture d'^ > also for q > 1. 

III. THE METHOD 

Suppose we are given two time series measured in sub- 
systems U\ and U2 of system U whose structure and in- 
terdependence wc do not know, e.g. signals gathered 
on two electrodes placed in not too far away portions of 
brain, or measurements of velocity or temperature in var- 
ious parts of moderately turbulent fluid. We would like 
to know, if the equations governing the dynamics of both 
of these variables are coupled or not, how many degrees 
of freedom are common and what is the direction of the 
coupling. 

Let Xi be a function on U, i.e. Xj : U — > K. The 
time series we measure are x\(n) := Xi(ni(t n )) and 
x 2 (n) := X 2 (u 2 (t n )). Let Y : R 2 — > R be a smooth func- 
tion nontrivially depending on both variables^]. We con- 
struct another time series y(n) — Y(x\(n),X2(n)). Thus 
Y(Xi,X2) is a function on U. 

Using time delay method we can reconstruct the 
dynamics of the systems U and U from Xi(n) and y(n). 
Namely, for a given delay r and embedding dimension N 
we construct delay vectors 

ui(n) = (xi(n),xi(n - r), . . . ,xx(n — (N - 1)t)); 

the construction of u 2 from x 2 and u from y is similar. 

If N > 2Dq(hi), for all reasonable delays, for infinite 
not-too-sparsely probed time series, the Takens theorem 
[^|,D guarantees iii(n) is an embedding of the original 
invariant set in U\. To calculate dimensions it is even 
enough to take N > D (^i) It is generally be- 

lieved that also for finite but not too short and not too 
noisy time series the above construction gives occasion- 
ally a reasonable estimate on the original dynamics. For 
a detailed discussion of these issues the reader should 
consult the relevant literature, e.g. ]igQ -Eg j . We dis- 
regard the practical problems until sectionM where we 
show some numerical results. For the time being we dis- 
cuss clean infinite time series. 



3 For finite noisy time series some functions are better than 
other. In practice we used five different functions Y(x,y), 
namely x + y, x-y, sin(a:) cos(y), xexp(y), 2x — y, to calculate 
dimension of the system D q (fiu), and averaged the results. 
The variance of the obtained five estimates was usually small. 

The above functions were not chosen for their particularly 
good numerical properties but rather to verify that the results 
obtained depend only weekly on the choice of the function Y. 



Having reconstructed the attractors we can estimate 
their generalized dimensions and calculate the generalized 
dimensions of interaction 

< := D q {in) + D q (^) - D q {nu). (8) 

It is also convenient to consider normalized dimensions 
of interaction: 

m\ := df/DM, 

ml := df/DM, (9) 

m q v := df/D^u). 

From the values of m| we can infer the information we 
need. All the possible cases are described in the next 
section. Note that if mj ^ 0, they satisfy 

1 1 1 _ 

m\ m\ m,jj 

From (|) we also have 

< mj < m\,m\ < 1, 

which provides us with a tool to check consistency of 
results. 

Before we present the classification of all the possible 
schemes of interaction let us discuss heuristically four 
simple examples. 

I If U\,U2 are uncoupled, the variables we see 
through Xi,X2 are different, thus fly = Mi x M2- 
Therefore, from Theorem 1, d'^ = 0, as it should 
be for any reasonable definition of dimension of in- 
teraction for non-interacting systems. 

II Consider now a system U consisting of three 
isolated systems Vi, which wc cannot observe 
separately, however, but rather through U\ and 
U 2 , e.g. measuring -X"i(vi,v 2 ) and X 2 (v 2 ,v 3 ). 
Reconstructing dynamics from time series of X\ 




FIG. 1. Simple interaction 



and X2 we expect to obtain 

D q (pi) = D q (fj, Vl ) + D q (nv 3 ), 
D q (n 2 ) = D q (fj, V2 ) + D q (nv 3 ). 

With a typical function Y(xi,X2) we obtain time 
series y(n) from which we estimate 
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D q {nu ) = D g Vi ) + D g (nv 2 ) + D q {n Vs ) . 

Since dynamics of V2 is responsible for the interac- 
tion between U\ and U2, we want to call the dimen- 
sion of interaction dimension of fi\/ 2 ■ According to 
the definition (j|) we have 

d' q nt = DM + DM - DgQiu) 

= D q (M Vl ) + Dg (fJ, V 2 ) + Dg (ft V 2 ) + Dg ) + 

-[DgiwJ + Dg(fi V2 ) + D q (fj, V3 )] 
= D q (fj, V2 ). 

Ill Con sider now the general situation described in sec- 
tion II C . Reconstructing dynamics from time se- 
ries of typical variables from systems U± and U2, 
say x\(n) and X2{n) we get 

DM > D q (fiv), 

Dq(fi 2 ) > DM)' 

For a typical function Y{x\,x 2 ) we obtain 
max{fl,(^i),I),(/i2)} 

< DM) 

< DM) + - DM)) + 
(dm-dM))> 

where D q {ji\) — D q (fiy) quantifies number od de- 
grees of freedom in U\ not coupled to U 2 . From 
this we conclude 



< Dg(pv) < df 



— ~q 

= dm + D M - d qM 

< min{£» g (/ii),Dg(^ 2 )}, 

the difference between D q (fj,v) and d' q nt depending 
on the strength of synchronization between U\ and 

IV As the last example we shall take a system X driv- 
ing two response systems Y\ and Y 2 . Suppose we 
also have a second copy of this setup, namely drive 
X' with response systems Y{ and Y" 2 '. We collect si- 
multaneously four time series of some variable from 
all of the response systems. Now we choose ran- 
domly two of them and want to know if the systems 
they come from had a common driver. 

It is easy to check that if they had, then d'^ is ap- 
proximately the dimension of the driver D q {^x) > 
0. If they had different drivers, then d'g* = 0. 

Summarizing, from measurements involving parts of 
the given system and arbitrary nontrivial smooth func- 
tion of two variables we can reconstruct the dimensions 
of measures fi%, /12 and fiu- From this we can obtain the 
dimension of interaction d'"* (||). Depending on the val- 
ues of Dg(fii), Dq(n2),D q (fiu) and ci' nt we can find out if 
the systems are coupled or not, and what is the direction 
of coupling. 



IV. CLASSIFICATION OF POSSIBLE 
INTERACTION SCHEMES 

Let us thus assume that we have two subsystems and 
the reconstructed dimensions for them are D q {p,i) and 
D q (n2)- The dimension of the whole system D q {fj,u) is 
obtained from time series y(n) constructed through the 
procedure described in the previous section. The dimen- 
sion of interaction is calculated from (||). The above dis- 
cussion leads to a question which situations are possible. 
There are four non-equivalent cases, which are conve- 
niently described by the following proposition 

Proposition 4. 

1. If d' q l = 0, then \x\j = /ii x /i 2 (the systems U± and 
U2 do not interact); 

2. If Dg(fii) = D q (fi 2 ) = d'^, then = /ii = fi 2 = 
fj,v (the systems U\ and U2 are the same system or 
we have maximal coupling) ; 

3. If Dq(/xi) > D q (p,2) = d!^, then ^ 2 = (iv and 
fix = fijj (all variables of U2 couple to some of the 
degrees of freedom ofU\, or U2 is the driver in the 
pair driver — response which is U\ = U ); 

4- In all the other cases Dg(pi), D q (p,2) > d' q l , which 
means interaction or double control (two response 
systems driven by a common driver). 

Note that this proposition is to some extent opposite 
to the theorems proposed in Section |n|. It can be shown 
for q = 1 We verify it numerically for particular 

systems for q — 2 in the next section. 

It is convenient sometimes to use m\ , m\ , (0) . We 
can write the above classification in this case as follows 



2. 1 = m\ = m\ = ray means maximal coupling: 
/ii = H2 = Hv; 

3. 1 = m\ > m\ = rnfj > means coupling of all the 
degrees of freedom of U2 = V to some variables of 

4. 1 > m\ > m\ > m v > means interaction or 
double control (two systems driven by a common 
driver) ; 

All the four cases are presented symbolically on Figure ||. 

The examples considered in the previous section can 
be easily identified as particular cases of this classifica- 
tion. Namely example I represents case 1, example II 
represents case 4, example III can represent cases 2, 3 or 
4, the last example represents cases 1 or 4 depending on 
whether the signals analyzed come from systems coupled 
to the same driver or not. 
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df = 



m\ = m\ = m\j = 
D q (jix) = D q {fi 2 ) = df > 

Q Q Q i 

?TiJ = 777,2 = m ?/ = 1 

D q (n2) > D q (m) = 4 nt > 
1 = mj > raf = > 

^(^ 1 ),i? g (M2)>< t >0 

1 > > m' > > 



FIG. 2. Classification of possible interaction schemes. The 
first column shows symbolically the relative position in the 
phase space of the subsystems in which we measure the time 
series x\ and X2- An arrow from one system to another means 
the future states of the second system depend on the current 
states of both. The second column shows the values of nor- 
malized dimensions ml, m\ and mj in each of the cases. 



V. NUMERICAL RESULTS 

Below we shall present some applications of our 
method to analysis of numerical results for several 
paradigmatic systems (coupled Henon maps and logistic 
maps). 

Throughout this section we will use d' 2 nt . The dimen- 
sions presented in the pictures are always D 2 calculated 
with the help of d2 program from TISEAN package ]49|] 
with an algorithm which is an extension of algorithms 
published previously |22|,|23|,|30| which improves speed of 
computation |^9| . In every case we used 10 5 points with 
one exception described in the text. The functions Y 
used to calculate the dimension of the whole system (cf. 
previous sections) were x + y, x-y, sin(x) cos(y), xexp(y), 
2x—y. To estimate the dimension we used Takens-Theiler 
estimator Jl0| , [49| pl| c2t and c2d smoothed output from 
d2. 

Typical behavior of local dimension d log C(e)/d log e 
as a function of resolution e is shown in figure 3. 



A. Two Henon maps 

Consider a system U consisting of two Henon maps j55 
coupled as follows JT^ ]: 



Local dimension of U, 
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FIG. 3. Takens estimator of correlation dimension. Data 
shown come from two different Henon systems driven by the 
third (eq. [H] with parameters B± = 0.3, Ci = 0.5, B2 = 0.1, 
C2 = 0.6.) . Correlation dimension estimated from the pic- 
tures is 1.95 for both of the subsystems, and 2.4 for the 
whole system. We show two plots out of five used to esti- 
mate the last number. Dimension of interaction in this case 
is 1.95 + 1.95 — 2.4 = 1.5 > 1.22, which suggests partial syn- 
chronization of the two response systems with the driver. 



G 



Ui+1 
Vi+1 = u 



1.4 - (Cxi + (1 - C)it 4 )w 4 + Bvi 



(10) 



Thus Henon system K drives system L. The coupling 
is introduced through variable u. We consider the case 
of coupled identical systems (B = 0.3) and non-identical 
coupled systems (B = 0.1). Parameter C measures the 
strength of interaction. 

Suppose the variables accessible experimentally are x n 
and u n . What can be said in this case about the inter- 
action between systems K and LI 

Certainly, for C — the systems K and L do not in- 
teract (case 1. in our classification), therefore D q (fiu) — 
Dqiidx) + Dqi^L) and d'^ = 0. On the other hand, for 
positive C the influence of x should reflect in the be- 
havior of u. From Theorem 3 we expect DJJ 1 * = D q (/j,x) 
(case 3.). One can also expect for C raising slightly above 
0, D q (fiu) not to change much, while D q (fiL) should 
jump from its value at to the value of D q {p,jj) at c = 0. 
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Dimensions of coupled Henon maps for B=0.1 
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FIG. 4. a) Dimensions D2(p.i), ^2(^2), DzUw) and d 2 nt of 
one-way coupled non-identical Henon maps (hoi) B = 0.1. b) 
Normalized dimensions mf, ml and mfj for the same systems. 
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one-way coupled identical Henon maps (tol) j3 = 0.3. b) Nor- 
malized dimensions ml, m\ and for the same systems. 



This behavior can indeed be seen in figure |4ja for non- 
identical Henon systems (B = 0.1) and in||a for identical 
systems (B ~ 0.3). The synchronization of x and u |5^] 
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visible for C > 0.7 (case 2.) can be discovered much 
simpler, namely if one plots several consecutive values of 
(100, say) versus coupling, for these particular 
values all of the points fall on (cf. figure 7 of jl2| ) . 

Looking at the normalized dimensions (fig. |b and |b) 
we easily identify lack of coupling for C — (mi = ra-i — 
TO3 = 0), case 4. (maximal coupling) for B = 0.3 and 
C > 7 (mi = TO2 = W3 = 1), and case 3. in all the other 
cases. 

The drop-down of the dimension at 0.7 for identical 
systems is connected with the full synchronization of the 
systems. The equations (|l^) admit solutions symmetric 
in x and u (x n — u n = 0), which at this region become 
stable and the whole probability measure gets localized 
on a lower-dimensional manifold. For more details cf. 



B. Three Henon maps 

Consider now the system U consisting of three Henon 
maps p3 coupled as follows E3j: 

K f x i+1 = 1.4 - zf + 0.3^, 
\ Vi+i = Xi, 

L (u i+1 = 1.4 - {CxXi + (1 - Cx)ui)ui + BiVi, 

\ V i+ l = Ui, 

( n„- , — 1 ZL — tr<„T- A- IA — n^nnAi,,- 4- R„r- 

M 



Thus Henon system K drives systems L and M. The 
coupling is introduced through variables u and w. Pa- 
rameters Ci, C2 measure the strength of interaction. 

Suppose the measurements on (K, L, M) yield vari- 
ables u and w. What can be said in this case about 
the interaction between the systems L and M? 

For C\ — C2 — neither L nor M systems feel the in- 
fluence of K. They also do not interact (case 1.). When 
one of Ci grows, the influence of K is immediately mir- 
rored in the rise of the dimension of \xl or fiM- For both 
Ci > the systems L and M interact (case 2.), and the 
part responsible for interaction is K. Thus the dimension 
of the common part is constant and equal to 1.22 in our 
case. 

We show this behavior in figures ^a for different param- 
eters (B\ — 0.3, B2 = 0.1) and ^a for the same parame- 
ters (Bi = 0.3, B2 = 0.3). In both cases C\ — 0.5 and Ci 
is varied. In both figures one can clearly see the jump of 
the dimension of interaction from to values equal to or 
greater than 1.22, the dimension of the attractor of K. 

Figure ^ is particularly interesting, since one can ap- 
parently identify all the four cases from our classifica- 
tion. For C2 = we have non-interacting systems, for 
C 2 € [0.2, 0.4] and C 2 = 0.6 we have case 2. 

For C2 = 0.5 the two Henon systems L and M become 
identical. Since at this value of coupling constant they 
are in general synchrony with the driver, which means 



(a) 



Dimensions of coupled Henon maps for B^O.3, C-|=0.5, B 2 =0.1 
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FIG. 6. a) Dimensions £>2(Mi)i 02(^2), D2dj.11) and d'™ of 
two-way coupled Henon maps ( |ll| ) with different response sys- 
tems (Ci = 0.5, B± = 0.3, B 2 = 0.1). Additional line at 1.2 in 
the upper figure stands for the dimension of the attractor of 
Henon system K. b) Normalized dimensions m\, m\ and m\j 
for the same systems. 
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their asymptotic states are independent of their initial 
states, and depend only on the present state of the driver, 
it follows that u n = w n . 

For C2 > 0.7 the system M fully synchronizes with K, 
which leads to the collapse of the probability measure in 
K, M space on the diagonal (compare the discussion in 
the previous subsection). 



C. Logistic maps 



Dimensions of coupled Henon maps for B^O.3, C-|=0.5, B 2 =0.3 
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FIG. 7. a) Dimensions £>a(^i). 1)2(112), D2(nu) and d'^ 1 of 
two-way coupled Henon maps (111]) with identical response 
systems (Ci = 0.5, Bi = 0.3, B 2 = 0.3). Additional line at 1.2 
in the upper figure stands for the dimension of the attractor 
of Henon system K. b) Normalized dimensions m\, ml and 
m\j for the same systems. 



Let f a (x) := ax(l — x). Consider a system consisting 
of four uncoupled logistic maps 



where ot\ = 3.7, a-i = 3.8, = 3.9 and 04 = 4. Suppose 
the only variables available experimentally aref] Y 1 ^ (n) — 
F 1 ' 3 \x % n , x^), i < j . Given two randomly chosen time 
series Y l ' 3 (n), Y k ' l (n) we want to know if they share some 
degrees of freedom or not (if they "interact" or not). If i 
or j is equal to k or I, there are only three active degrees 
of freedom in the compound system. Otherwise there are 
four. 

Estimated correlation dimensions for several cases are 
collected in Table |[ In every case we used time series 10 5 
points long except for the last one, for which 10 6 points 
were used. The estimation error was roughly 2% except 
for the last case for which it was about 5-10%^. 

Consider now two symmetrically coupled logistic maps 



Vn+1 = ffiiVn), 



where 



x n 
Vn 



1+c ' 

y n +cx n 

1+c ■ 



(12) 



and parameter c G [0, 1] measures the coupling. This is 
slightly different from couplings discussed previously in 
the literature (e.g. p4]-pti[|). The maps are uncoupled for 
c = 0. For c = 1 (the strongest coupling) if we set z n := 
x n = y„, we have x n = -§^z n , y n = ^z n , and z n+1 = 

f a+i3 (z n ). Therefore dynamics is one-dimensional. Case 

2 

c > 1 is equivalent to c' = 1/c. 

Estimated correlation dimensions for several values of 
the coupling constant c are shown in figure |^. One can see 
the jump in the dimension of interaction from at c = 
to the value equal to the dimension of the whole systems 



4 The coupling functions F 1 ' 3 were chosen randomly out of 
x + y, x-y, sin(x) cos(y), x exp(y), 2x - y. 

J We believe there are two reasons for this. One is higher 
dimensionality of the system in the last case (four uncupled 
logistic maps), the other is worse ergodicity in the phase space 
because the maps are uncoupled. Note that our procedure 
consists of two parts: first we make the embedding, then we 
calculate the dimensions. Each of the two can introduce er- 
rors. The number expected in the last case is the sum of the 
first four numbers, namely 3.87 
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series x(n) 




x 1 


0.96 


X2 


0.95 


%3 


0.97 


X4 


0.99 


yl,2 


1.88 


y 1,3 


1.94 


yl,4 


1.95 


y2,3 


1.89 


Y 2 ' 4 


1.94 


y3,4 


1.93 


f(Y 1 ' 2 ,Y 1 ' 3 ) 


2.88 




3.8 



for positive c indicating case 4. in our classification. For 
c = 0.2 asymptotic dynamics settles on a periodic orbit 
leading to all the dimensions equal to 0. Numerically ob- 
tained approximations to asymptotic measures for cou- 
pling constant c = 0., 0.1, 0.2, 0.3, 0.4, 0.5 are shown in 
figure 10. Note the increasing synchronization between 



x and y. 

It is of interest to compare the values of dimensions for 
c = and 1, because in both cases Di([i x ) « Dx(fx y ) = 1, 
but the dimension of the whole system, estimated from 
f(x n , Vn) is equal to 2 in the first case, and 1 in the sec- 
ond, implying D lnt = and 1 in these cases, respectively. 
Thus the first measure has a product structure, while the 
other is concentrated on the diagonal x = y. 



FIG. 8. Estimated correlation dimension for uncoupled lo- 
gistic maps. The estimation error is roughly 2% except for 
the last number for which it is about 5-10%. 
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FIG. 9. a) Dimensions I?2(/ii), ^(a^), D iifJ-u) and d™ 1 of 
symmetrically coupled logistic maps (112). b) Normalized di- 
mensions mi, mj and for the same systems. 
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FIG. 10. Attractors of symmetrically coupled logistic maps 
(||) for c = 0, 0.1, 0.2, 0.3, 0.4, 0.5 in (x, y) plane. 



The last case considered is that of the double control: 
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X n +1 
Vn+1 
Zn+l 



fot\Xn)i 

fp(V-n.) + ClXr. 
1 + Cl 

1 + C 2 



(13) 



where a = 4.0, j3 = 3.8, 7 = 3.9, c 1; c 2 £ [0,1]. Let 
the observed systems U\ and U 2 be the sets of all pairs 
(x,y) and (x,z), respectively. Then we have essentially 
the case 2. If JJ\ and U2 are the sets of all points x and 
pairs (x, z) then we have the case 3. 



(a) 



Asymmetric coupling: x and z, a=4.0, y=3.9 



2 - 



E 

b 



1.5 



0.5 



D 2 (Hl) — *- 
D 2 (n 2 ) -~ »- 

D 2 (M 



(b) 



0.2 0.4 0.6 O.i 
Coupling constant C 2 



Normalized dimensions m 



1 




0.2 0.4 0.6 0.8 1 

Coupling constant C 2 

FIG. 11. a) Dimensions D2(/ii), 1^2(^2), D2(fJ.u) and d 2 nt of 
asymmetrically coupled logistic maps ( |l3| ) when a; and 2 are 
the observed variables, b) Normalized dimensions m\ 
and mfj for the same systems. 
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Figures [fl] and |l2| show estimated correlation dimen- 
sion in these cases. Again, one can clearly see the differ- 
ence between the coupled (ci > 0) and uncoupled (c, = 0) 
systems, because the interaction dimension jumps from 
to 1 or more, in agreement with our expectations from 
theorems 2 and 3, since the dimension of the common 
part is 1 (x n evolves according to Ulam map: a = 4.0). 
Figure |l^ shows projections of the attractor of ([l3|) on 
(x, z) and (y, z) planes for c\ = 0.1 and C2 = 0.2. 
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Asymmetric coupling: y and z, a=4.0, (5=3.8, y=3.9, c^O.1 
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FIG. 12. a) Dimensions Z?2(mi)> -^Mmi), Dzinu) and dj"' of 
asymmetrically coupled logistic maps ( |l3| ) when y and z are 
the observed variables, b) Normalized dimensions mf, m\ 
and rnfj for the same systems. 
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FIG. 13. Projections of the attractor of asymmetrically 
coupled logistic maps (|l3| ) for ci = 0.1, C2 = 0.2 on (x, y) 
and (y, z) planes. 
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VI. CONCLUSIONS AND OUTLOOK 



APPENDIX A: THE PROOFS. 



We presented a method which allows to distinguish in- 
teracting from non-interacting systems when time series 
of variables of the two systems are available. Partial proof 
of its validity was provided. Classification of all the pos- 
sible interaction schemes was presented with examples 
of all the cases. Several simple interacting systems were 
analyzed. 

To use our method in practice (from field data) we 
suggest the following procedure: 

A. calculate the dimensions D q (fii), D q (fi2), D q (nu) 
and d q l (]|[) (we suggest q = 1 or q = 2; it is also 
good to normalize the data if they are od different 
orders); 



B. repeat the calculation for several different coupling 
functions Y and average the results (linear combi- 
nation seem to be the best choice); 

C. if they are different from 0, calculate the normalized 
dimensions mf @. 

D. they may take one, two or three distinct values. 

(a) if all of them are 0, the systems do not interact 
(case 1.); 

(b) if all of them are greater than and less than 
1, this is a generic case of interacting systems 
(case 4.); 

(c) if one of them is 1, the other are smaller, all 
the degrees of freedom of one system couple to 
some degrees of freedom of the other (case 3.), 
or we have the previous case (case 4.) but the 
variables of one of the systems which are not 
coupled to the other synchronize to the system 
comprising the common part of the dynamics; 

(d) if they are all equal to 1, all the degrees of 
freedom of one system couple to all the de- 
grees of freedom of the other (case 2.), or we 
have the two previous cases (3. or 4.) but 
the variables of the two systems which are not 
coupled synchronize to the system comprising 
the common part of the dynamics. 

Our method has been successfully used to distinguish 
between interacting and non-interacting Chua systems in 
an experiment |57]]. We hope it shall prove a useful tool 
in analysis of other complex systems. 
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Let /xi, (U2 be the i nvari ant measures of systems Ux, U<i 
as defined in Section El] 



Theorem 1 Suppose D q (/ii), D q (/i2), D q (l^i x ^2) ex- 
ist. Then 

D q (pi x /x 2 ) = Dqinx) + D q (p 2 )- 

Proof: Take q ^ 1. For every e > consider partitions 
of R rai into cells of volume e Ui . This gives a partition in 

R n 1+ n 2 intQ boxes of volume £ n 1+ n 2 _ 

Let 

Pj = — th cell from the cover of Ux), 
i"k = ^%{k — th cell from the cover of U%). 



Then 



D q (pi x (is) = hm — - 

q e^o q-1 loge 



lim 



1 log (E*p?) 



= lim 



q — 1 log e 

-*o \ q — 1 log e ) 

1 lo sE^A 



lim . . 

e^Q \ q — 1 loge / 

But the last two limits exist and are equal to £> 9 (/ii) and 
D q {^2), respectively. 

The case of q = 1 is straightforward and left to the 
reader. □ 

For the next proof we need the following Lemma. 

Lemma 5 Let 1 > c tj > 0, J2ij °ij = ^ a i = J2j c ih 

b i = EiCy- Then 



J ^2(a i bj log(a t bj) - c y logc^ ■) < 0. 



(Al) 



1,3 



Proof: Every convex function / satisfies Jensen's in- 
equality 



(A2) 



where EiP« = 1- Since f(x) — 2 log 2; is convex, one has 

,(y>) <E°a/(2-). 

\ »j / 1,3 

f(l) < ^2 aib j—j- (log Cij - loga t - bj) 



h3 



< (Hj log Cij ~ ^ c ij lo S a i~^Z c% i log b i 
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i,j i j 

Cij log - ajbj log(ai6j)), 

where we took pij = aibj and a;^- = Cij / {aibj). □ 

Let /Xx, M27 M $ an d Ms De the invariant measures de- 
fined in Section |ll C| . 

Theorem 2 Suppose D\{^\), Di{^2), Di(^iv), 
Di(pu) exist. Then 

Di(fiv) < d, nt ■= £>i(Mi) + Di(fi 2 ) - Di(nu). 

(We shall call d lnt dimension of interaction ). The equality 
holds when y\ and y 2 are asymptotically independent. 

Proof: There are ni+n 2 independent variables thus the 
system can be embedded in K ni+ " 2 . Consider a partition 
of ffi™ 1+ ™ 2 into cells of size e consistent with the structure 
of equations of dynamics, i.e. (i,j, k)-th cell= Ai x Bj x 
Cfc, where A,B,C are e-cells of dimension, respectively, 
k\ + k 2 , n i ~ k\, n 2 — k 2 hi spaces spanned by x, yi and 

Since the dynamics of (x, yi ) is independent of y2 , the 
invariant measure fii(Ai x Bj) can be written as 

fii(Ai x B.j) = nviA^my^iBjlAi) =: piTji, 

where fjLi Vl \ x \{Bj\Ai) are the conditional probabilities of 
finding the yi in Bj under the condition x being in A4. 
Similarly, 



Of course, 



fj, 2 (Ai x C k ) = Hv{Ai)ii( y2 \ x ){C k \Ai) =: pis ki 



and 



Hs(Ai x Bj x C k ) = Hv(Ai)fj,(y uya \ x )(Bj,Ck\Ai). 

— . Pitjki 

If fir yi \ x \(Bj\Ai) and /j,( y2 \ x }(Ck\Ai) are independent, 
then 

Hvi,y2\x)( B j> G k\Ai) = M( J/1 | :c )(-B i | J 4 i )/i(2 /2 | a: )(Cfc|yli), 

(A3) 

otherwise the only thing we know is that the l.h.s. mea- 
sure is the coupling of the r.h.s. measures, namely 

^ /%i , V 2 1 x) {Bj , C k I Aj ) = H( yi \ x -)(Bj\Ai), 
k 

X] ^{vuvi\x){Bj,Ck\Ai) = tX(y 2 \ x )(C k \Ai), 



^ 1 tjki — r J'i' 



if Pi ^ 0. Otherwise we take Vj, fc : fjjy = 0. 

Taking this into consideration, inequality (||) follows: 

e,e ; m<o< 'OR* /'.'•..,' 



lim 

lim 

£^0 



lim 



lim 

£^0 



loge 
loge 
loge 

loge 



= lim 



lim 



loge 

E;ft Ej,k( r ji s M log^Sfej) - t j7W log^-fei)) 
loge 



> 



where in the last line we used Lemma 5 for c = t, a 
and b — s and the fact that loge < 0. 

Note that the equality holds if and only if 



tjki rjiSki • 



(A4) 



This is what we call asymptotical independence of vari- 
ables yi and 2/2 ■ In particular, when yi are in generalized 
synchrony with x, this means that their asymptotic be- 
havior is independent of their initial states and depends 
only on initial state of x, therefore their probability dis- 
tributions cannot be independent, since they depend on 
the same number a:(0). However, we are not sure if this 
the only case when the equality is not satisfied, this is 
why we use another name for the above condition. □ 

One would like to establish a similar inequality in case 
of other Renyi dimensions, however, in general, even 
when (A4) is satisfied, 



D q (vs) D q {»i) + DM - D q (fi x )- 

Indeed, 

D q (fli) + D q (fi 2 ) - D q (fJ, x ) - Dg(ns) = 
1 



(A5) 



lim 



loge 

lim 

s^o lose 



log 
log 



(j2i,kPi r kij (j2LjPl s< jl 

(EipI) (E,,.a i'!*l't 
Y. l< i,j,kP q l p q i( r l l - r li)i. s % - s ji) 

q q q q 

2^,i,j,k,lPiPl S ji r ki 



This may have arbitrary sign and needs not vanish in the 
limit. 
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Although (A.5) must go to in the limit q — ► 1, one 
can perhaps construct examples of measures for which 
the slope can be arbitrarily large. On the other hand, we 
believe such measures will not be typically observed in 
physical systems. 



APPENDIX B: AN EXAMPLE OF PARTIALLY 
COUPLED SYSTEMS. 

We present here a simple example of interacting sys- 
tems for which one can introduce the natural decompo- 
sition (Q). 

Consider two systems U± , J7 2 interacting through a thin 
contact layer V. Denote variables in U\ as Ui = (vi,Wi), 
variables in U2 as 112 = (v2,W2), and variables of the 
contact layer V are (vi,V2). Dynamics of such a config- 



Wi 



Vl 



v 2 



w 2 



FIG. 14. Interacting systems, 
uration can be described as 

Wl = /i(vi, Wi), 

w 2 = /a(v2, w 2 ), 
vi = gi(vi, v 2 , wi), 
v 2 = 32 (vi, v 2 , w 2 ). 

If we can average the influence of wi , w 2 on the dynam- 
ics of Vi,v 2 , e.g. when the time scales involved in the 
dynamics of Vj and Wj are different, we obtain 

wi = /i(vi,wi), 
w 2 = / 2 (v 2 , w 2 ), 

vi = ,9i(vi,v 2 , Ai), (Bl) 
v 2 = 32<vi,v 2 , A 2 ), 

where Ai, A 2 measure the average influence of wi, W2 on 
V. Thus equations for vi, V2 comprise a closed system V. 
This part of dynamics is responsible for the interaction. 
Note that this scheme can also be considered as a dou- 
ble control configuration of three systems, where (vi, V2) 
control Wi and W2 . 

If we s et x := (vi,V2), y, = Wj, then the equa- 
tions (Bl) reduce to equations (Q). 
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